Un sistema multi-agente di supporto alle decisioni per lo smistamento di flussi croceristici in città

Estudi per trobar una solució als problemes de congestió produits per l'arribada de creueristes i la manera de minimizar-los utilitzant un Sistema de Suport a les Decisions basat en la metodologia dels Sistemes Multi-Agent

Evaluating the benefits of key-value databases for scientific applications

The convergence of Big Data applications with High-Performance Computing requires new methodologies to store, manage and process large amounts of information. Traditional storage solutions are unable to scale and that results in complex coding strategies. For example, the brain atlas of the Human Brain Project has the challenge to process large amounts of high-resolution brain images. Given the computing needs, we study the effects of replacing a traditional storage system with a distributed Key-Value database on a cell segmentation application. The original code uses HDF5 files on GPFS through an intricate interface, imposing synchronizations. On the other hand, by using Apache Cassandra or ScyllaDB through Hecuba, the application code is greatly simplified. Thanks to the Key-Value data model, the number of synchronizations is reduced and the time dedicated to I/O scales when increasing the number of nodes.This project/research has received funding from the European Unions Horizon 2020 Framework Programme for Research and Innovation under the Speci c Grant Agreement No. 720270 (Human Brain Project SGA1) and the Speci c Grant Agreement No. 785907 (Human Brain Project SGA2). This work has also been supported by the Spanish Government (SEV2015-0493), by the Spanish Ministry of Science and Innovation (contract TIN2015-65316-P), and by Generalitat de Catalunya (contract 2017-SGR-1414).Postprint (author's final draft

The OTree: multidimensional indexing with efficient data sampling for HPC

Spatial big data is considered an essential trend in future scientific and business applications. Indeed, research instruments, medical devices, and social networks generate hundreds of petabytes of spatial data per year. However, many authors have pointed out that the lack of specialized frameworks for multidimensional Big Data is limiting possible applications and precluding many scientific breakthroughs. Paramount in achieving High-Performance Data Analytics is to optimize and reduce the I/O operations required to analyze large data sets. To do so, we need to organize and index the data according to its multidimensional attributes. At the same time, to enable fast and interactive exploratory analysis, it is vital to generate approximate representations of large datasets efficiently. In this paper, we propose the Outlook Tree (or OTree), a novel Multidimensional Indexing with efficient data Sampling (MIS) algorithm. The OTree enables exploratory analysis of large multidimensional datasets with arbitrary precision, a vital missing feature in current distributed data management solutions. Our algorithm reduces the indexing overhead and achieves high performance even for write-intensive HPC applications. Indeed, we use the OTree to store the scientific results of a study on the efficiency of drug inhalers. Then we compare the OTree implementation on Apache Cassandra, named Qbeast, with PostgreSQL and plain storage. Lastly, we demonstrate that our proposal delivers better performance and scalability.Peer ReviewedPostprint (author's final draft

A staging area for in-memory computing

An in-memory staging area provides fast access to different applications. This research is based on evaluating the benefits of a distributed in-memory staging area applied to the field of Big data. With this purpose, a prototype is designed and proposed to verify the idea. Then, a working version comprised of the in-memory software Alluxio and the processing engine Apache Spark is deployed and evaluated. In particular, the work demonstrates the increase in performance resulting from updating the data in the in-memory staging instead of allocating space for new objects. The evaluation is conducted by running an analytic with Spark over a continuously changing dataset stored in Alluxio. The experiments reported a throughput increase of 10x when compared to storing information in a regular parallel filesystem, and an increase of 3x compared to the official deployment methodology. By updating the dataset, the Alluxio in-memory capacity stays constant at a low level compared to current deployments where its capacity decreases linearly, resulting in lower performance.Esta investigación se basa en evaluar los beneficios de un área de almacenamiento en memoria distribuida aplicada al campo de Big data. Con este propósito, se diseña y propone un prototipo para verificar la idea. Luego, se implementa y evalúa una versión de trabajo compuesta por el software de memoria Alluxio y el motor de procesamiento Apache Spark. En particular, el trabajo demuestra el aumento en el rendimiento que resulta de actualizar los datos en la puesta en escena de memoria en lugar de asignar espacio para nuevos objetos. La evaluación se realiza ejecutando una analítica con Spark sobre un conjunto de datos que cambia continuamente y que se almacena en Alluxio. Los experimentos sacaron un aumento del rendimiento de 10x en comparación con el almacenamiento de información en un sistema de archivos paralelo regular, y un aumento de 3x en comparación con la metodología de implementación oficial. Al actualizar el conjunto de datos, la capacidad en memoria de Alluxio se mantiene constante en un nivel bajo en comparación con las implementaciones actuales, donde su capacidad disminuye linealmente, lo que resulta en un menor rendimiento.Aquesta recerca es basa a avaluar els beneficis d'un àrea d'emmagatzematge en memòria distribuïda aplicada al camp de Big data. Amb aquest propòsit, es dissenya i proposa un prototip per verificar la idea. Després, s'implementa i avalua una versió de treball composta pel programari de memòria Alluxio i el motor de processament Apatxe Spark. En particular, el treball demostra l'augment en el rendiment que resulta d'actualitzar les dades en la posada en escena de memòria en lloc d'assignar espai per a nous objectes. L'avaluació es realitza executant una analítica amb Spark sobre un conjunt de dades que canvia contínuament i que s'emmagatzema en Alluxio. Els experiments van treure un augment del rendiment de 10x en comparació de l'emmagatzematge d'informació en un sistema d'arxius paral·lel regular, i un augment de 3x en comparació de la metodologia d'implementació oficial. En actualitzar el conjunt de dades, la capacitat en memòria de Alluxio es manté constant en un nivell baix en comparació de les implementacions actuals, on la seva capacitat disminueix linealment, la qual cosa resulta en un menor rendiment

Un sistema multi-agente di supporto alle decisioni per lo smistamento di flussi croceristici in città

Estudi per trobar una solució als problemes de congestió produits per l'arribada de creueristes i la manera de minimizar-los utilitzant un Sistema de Suport a les Decisions basat en la metodologia dels Sistemes Multi-Agent

Un sistema multi-agente di supporto alle decisioni per lo smistamento di flussi croceristici in città

Estudi per trobar una solució als problemes de congestió produits per l'arribada de creueristes i la manera de minimizar-los utilitzant un Sistema de Suport a les Decisions basat en la metodologia dels Sistemes Multi-Agent

An agent-based DSS supporting the logistics of cruise passengers arrivals

The arrival of cruises in a city represents an unmissable opportunity for the city economy to increment its tourist market penetration. Nevertheless, the management of an unforeseen number of passengers that need to visit the city in short time may have a negative impact on the city, so reducing the expected benefits. This is mainly due to the difficult in taking the right decisions when organizing the dispatching of passengers in different city areas, since these decisions depend on several conditions, that can also dynamically occur, and may impact different city sectors. In order to address the problem of organizing transportation and city tours for cruise passengers in a city, a Decision Support System is proposed to help both planning passengers transportation in the city, and also to evaluate the consequences for the city if the plans are really implemented. The system is designed according to the multiagent paradigm, so allowing to easily manage the necessary coordination among different entities and data sources that are usually involved in the considered application domain